Prediction Scores as a Window into Classifier Behavior

Most multi-class classifiers make their prediction for a test sample by scoring the classes and selecting the one with the highest score. Analyzing these prediction scores is useful to understand the classifier behavior and to assess its reliability. We present an interactive visualization that facilitates per-class analysis of these scores. Our system, called Classilist, enables relating these scores to the classification correctness and to the underlying samples and their features. We illustrate how such analysis reveals varying behavior of different classifiers. Classilist is available for use online, along with source code, video tutorials, and plugins for R, RapidMiner, and KNIME at https://katehara.github.io/classilist-site/.Comment: Presented at NIPS 2017 Symposium on Interpretable Machine Learnin

Bias Mitigation Framework for Intersectional Subgroups in Neural Networks

We propose a fairness-aware learning framework that mitigates intersectional subgroup bias associated with protected attributes. Prior research has primarily focused on mitigating one kind of bias by incorporating complex fairness-driven constraints into optimization objectives or designing additional layers that focus on specific protected attributes. We introduce a simple and generic bias mitigation approach that prevents models from learning relationships between protected attributes and output variable by reducing mutual information between them. We demonstrate that our approach is effective in reducing bias with little or no drop in accuracy. We also show that the models trained with our learning framework become causally fair and insensitive to the values of protected attributes. Finally, we validate our approach by studying feature interactions between protected and non-protected attributes. We demonstrate that these interactions are significantly reduced when applying our bias mitigation

A Task-Based Evaluation of Combined Set and Network Visualization

This paper addresses the problem of how best to visualize network data grouped into overlapping sets. We address it by evaluating various existing techniques alongside a new technique. Such data arise in many areas, including social network analysis, gene expression data, and crime analysis. We begin by investigating the strengths and weakness of four existing techniques, namely Bubble Sets, EulerView, KelpFusion, and LineSets, using principles from psychology and known layout guides. Using insights gained, we propose a new technique, SetNet, that may overcome limitations of earlier methods. We conducted a comparative crowdsourced user study to evaluate all five techniques based on tasks that require information from both the network and the sets. We established that EulerView and SetNet, both of which draw the sets first, yield significantly faster user responses than Bubble Sets, KelpFusion and LineSets, all of which draw the network first

Interactive visual analysis of relational data and applications in event-based business analytics

Zsfassung in dt. SpracheIm Zuge der hier präsentierten Arbeit wurde ein System entwickelt, um attributierte Graphen zu analysieren. In diesen Graphen werden Attribute zu den Knoten und den Kanten zugeordnet. Das System bietet mehrere Dartellungen für die visuelle Analyse im Graph an, sowohl auf einem lokalen als auch einem globalen Niveau. Die lokale Analyse erlaubt eine Navigation im Graph um die Struktur und die Attribute von einem Teil des Graphs zu erfoschen. Die globale Analyse erlaubt, die Verteilungen der Attribute in dem graph (oder im ausgewälten Teil vom Graphen) zu verstehen, und bestehende Assoziationen zwischen den Attributen und den Relationen festzustellen.Die Arbeit präsentiert eine Reihe von Erweiterungen zu den Attributen, die beispielsweise, graphtheoretische Merkmale, aggregierte Werte und hierarchische Gruppierungen repräsentieren können. Diese Attribute werden gleich wie die intrinsischen Attribute behandelt, und können mit den selben Methoden analysiert werden.Zusätzlich präsentiert die Arbeit neue Methoden für das Zeichnen von Graphen, die speziell für attributierte Graphen geeignet sind, insbesondere um die Verteilung und Assoziation von Attributen zu betonen. Diese Methoden können sowohl für die Navigation im Graphen als auch für die Darstellung von Abfrageergebnissen angewendet werden.Des weiteren erfoscht die Arbeit mehrere Arten von Assoziationen in elationalen Daten, und bietet Lösungen für die visuelle Analyse von diesen Assoziationen an. Dafür werden Erweiterungen für die bekannte "Parallel Sets"-Technik präsentiert, um die Perzeption zu verbesssern, und zusäzliche Informationen von den Daten einzubeziehen. Darüber hinaus werden neue Methoden für anderen Arten von Assoziationen vorgeschlagen.Die präsentierten Techniken sind generalisierbar und lassen sich auf beliebige Ereignisdaten anwenden. Das System wurde anhand zweier realer Datensätze auf seine Einsatzfähigkeit geprüft. Im ersten Datensatz werden aus einem Ereignis "Produkt gekauft" die Entitäten "Kunde" und "Produkt" abgeleitet samt ihrer Attribute (wie Kundenwohnort u.Ä) und Relationen (die durch die Historie der Käufe entstehen). Das System erlaubt in der Folge eine Reihe von Analysen um beispielsweise festzustellen, welche Arten (Kategorien) von Produkten vermehrt von Damen gekauft wurden, oder welche Altersgruppe besonderes Interesse für Bücher hat. Im zweiten Datensatz werden Ereignisse aus einem sogenannten Issue-Tracking-System analysiert um festzustellen, wie Support-Fälle bearbeitet und auch zwischen Support-Büros delegiert werden.In this work, a framework for interactive visual analysis of attributed graphs has been developed. An attributed graph is an extension of the standard graph of a binary relation, which attaches a set of attributes to the nodes and edges. The implemented visual analysis techniques aim at the local level at enabling an intuitive navigation in the graph which reveals both the structure of the selected part of the graph and the attributes of the nodes and edges in this part. At the global level these techniques aim at understanding the distributions of the attributes in the graph as a whole or in specific parts in it and at spotting meaningful associations between the attributes and the relations. The work presents several extensions to the attributes such as graph-theoretic features, values aggregated over the relations, and hierarchical grouping. All attributes are treated in a unified manner which helps performing elaborate analysis tasks using the existing tools. Additionally, novel graph drawing techniques are proposed. They are designed to understand attribute distributions and associations in the graph. These techniques can be additionally used to visualize results of queries in the data, which can be also visually defined using the attribute analysis tools. Finally, the work addresses several types of association analysis in relational data, along with visual analysis methods for them. It presents a perceptual enhancement for the well-known parallel sets technique for association analysis in categorical data, and proposes extensions for employing it in relational data. Also, novels methods for other types of association analysis are introduced. The relational data in this work were defined upon typed events in an event-based system, which offers a flexible architecture for real-time analysis. Nevertheless, the presented analysis methods are generic and have been tested on two real-world datasets. In the first dataset, entities for customers and products are derived from the purchase events, and various meaningful associations were found between the attributes and the relation (for example, which types of products the female customers bought more frequently, or at which age customers have higher interest for books). In the second dataset, events in an issue-tracking system are analyzed to find out ticket assignment patterns and forwarding patterns between the support offices.12